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FOREWORD 


The  Logistics  Training  Technologies  Technical  Area  of  the  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  is  engaged  in  research 
and  development  to  increase  Army  Training  effectiveness  by  implementing  im¬ 
proved  instructional  methods  and  advanced  technology.  To  meet  this  require¬ 
ment,  a  cooperative  Training  Technology  Transfer  Program  has  been  established 
with  the  U.S.  Army  Quartermaster  School,  Fort  Lee,  VA.  The  objective  of  this 
program  is  to  identify  and  test  candidate  training  technologies  and  strategies 
and  to  institutionalize  those  that  prove  effective  for  Quartermaster  skills 
training. 

This  report  presents  a  method  to  plan  and  evaluate  the  automation  of  Army 
classroom  training  and  us?s  the  Equipment  Records  and  Parts  Specialist  Course 
for  a  case  study.  It  illustrates  how  cost  and  effectiveness  data  may  be  used 
to  target  computer  applications  to  have  the  greatest  impact  on  training  time 
and  cos  i . 

This  report  was  made  possible  by  the  Congressional  Office  of  Technology 
Assessment,  which  made  the  author  available  on  an  interagency  detail  to  the 
Army  Research  Institute  to  accomplish  this  work. 
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COST-EFFECTIVE  AUTOMATION  OF  ARMY  CLASSROOM  TRAINING:  A  CASE  STUDY 


EXECUTIVE  SUMMARY 


Requirement : 

To  develop  a  cost-effectiveness  method  for  automation  of  Army  classroom 
training  and  to  apply  this  method  to  the  Equipment  Records  and  Parts  Special¬ 
ist  (76C10)  Course. 


Procedure: 

Average  costs  per  trained  soldier  were  collected  and  used  to  develop 
cost-effectiveness  strategy.  The  lock-step  training  process  in  the  course  was 
modeled  in  terms  of  marginal  (hour  by  hour)  cost  and  product  curves,  per 
classroom,  and  in  terms  of  different  training  costs  for  faster  and  slower  stu¬ 
dents.  Current  policy  for  automation  of  hands-on  training  was  reviewed  as  it 
increases  training  time  and  cost  and  as  it  reduces  costs  for  incremental  in¬ 
structional  applications  of  existing  classroom  computers. 


Findings: 

On  the  average,  it  costs  about  $7,500  to  train  an  Equipment  Records  and 
Parts  Specialist  for  10-11  weeks,  with  over  95%  of  costs  tied  to  course 
length.  This  suggests  that  the  most  powerful  way  to  reduce  costs  would  be  to 
shorten  training  time.  The  cost  breakdown  also  shows  that  technical  instruc¬ 
tion  accounts  for  only  14%  of  total  costs.  This  relatively  low-cost  share 
means  that  a  large  percentage  increase  in  cost  for  technical  instruction 
causes  a  much  smaller  percentage  increase  in  total  cost  and  that  improved  in¬ 
struction  may  reduce  almost  all  other  training  costs  by  shortening  training 
time. 


Each  day  of  training  time  saved,  via  improved  instructional  technology, 
saves  $425,000.  If  new  instructional  software  has  a  useful  lifetime  of  3 
years,  than  a  $1  million  investment  in  software  would  increase  total  training 
costs  by  less  than  2%,  while  it  would  Increase  inputs  for  technical  instruc¬ 
tion  by  about  33%.  Classroom  productivity  curves  are  used  to  target  instruc¬ 
tion  blocks  in  which  improved  instruction  is  most  likely  to  shorten  required 
training  time. 


Utilization  of  Findings: 

The  method  can  be  used  to  estimate  the  effects  and  costs  of  current 
classroom  training  and  future  automation  options.  With  appropriate  cost  and 
productivity  data,  the  analysis  targets  optimal  classroom  applications  of  com¬ 
puters  in  the  76C10  course  and  in  other  Army  courses  that  employ  lock-step 
instruction  and  use  computers  for  hands-on  simulation  of  automated  job  tasks. 
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COST-EFFECTIVE  AUTOMATION  OF  ARMY  CLASSROOM  TRAINING: 

A  CASE  STUDY 


INTRODUCTION  AND  SUMMARY 

The  paper  begins  with  a  review  of  average  training  costs,  per  soldier, 
for  the  Equipment  Records  and  Parts  Specialist  Course,  known  as  76C10,  at 
the  Quartermaster  School  (QMS)  in  Fort  Lee,  Virginia.  The  current  76C10 
Course  costs  about  $7500  per  graduate  for  10-11  weeks  of  training.  This 
average  cost  is  broken  down  into  major  cost  components  using  the  Cost 
Element  Structure  (CES)  Model  (Knapp  and  Orlansky,  1983).  The  CES 
breakdown  has  two  major  implications  for  cost  effectiveness  strategy. 

First,  since  course  length  determines  over  95X  of  cost,  the  most  powerful 
way  to  reduce  costs  would  be  to  shorten  the  Course.  Second,  since  costs 
for  technical  instruction  amount  to  a  small  share  (14X)  in  total  cost,  a 
large  percentage  increase  in  outlays  for  technical  instruction  accounts  for 
a  much  smaller  percentage  increase  in  total  training  cost.  At  the  same 
time,  if  new  instructional  inputs  shorten  required  training  time,  these 
increased  outlays  may  leverage  larger  savings  in  total  training  cost. 

Cost  effectiveness  methodology  is  presented  next.  Classroom  training 
in  the  76C10  Course  is  modeled  as  a  dynamic  production  process  in  which 
costs  can  be  attached  to  inputs  and  value  imputed  to  the  existing 
performance  criteria.  Classroom  cost  and  productivity  are  defined  hour  by 
hour.  Cost  structure  is  considered  first  because  the  data  have  been 
collected  for  administrative  purposes.  The  analysis  leads  to  the 
conclusion  that  saving  one  hour  of  actual  training  time,  for  all  75 
classes  during  one  year,  saves  about  $425,000.  Saving  one  day  of  training 
lowers  annual  training  costs  by  over  $1,000,000. 

Classroom  productivity  is  defined  as  the  number  of  soldiers  meeting 
the  current  test-based  performance  criterion  hour  by  hour  during  an 
instruction  block.  Productivity  varies  over  time  because  some  soldiers 
learn  faster  than  others.  Productivity  curves  vary  across  different  blocks 
of  instruction  because  some  subject  matter  is  easier  to  learn  than  others. 
Both  differences,  among  soldiers  and  subject  matter,  can  be  used  to  target 
when  and  where  automation  and  other  instructional  program  changes  may 
result  in  greatest  reductions  in  training  time  and  thus  result  in  greatest 
cost  savings. 

At  this  point,  the  discussion  shifts  to  policy  for  classroom 
automation.  Current  policy  dictates  that  classrooms  will  be  automated  to 
permit  hands-on  simulation  of  tasks  that  are  automated  in  the  field.  This 
policy  is  demonstrated  in  terms  of  a  3-week  extension  to  the  76C10  Course 
when  about  10X  of  Course  graduates  train  on  the  Tactical  Army  Combat 
Support  Computer  System/Standard  Army  Maintenance  System  (TACCS/SAMS) .  The 
costs  of  this  existing  automated  training  system  are  presented  in  terms  of 
the  Training  Cost  Breakdown  Structure  (TCBS)  (Seidel  and  Vagner,  1977). 

The  analysis  shows  that  automation  for  hands-on  simulation  marginally 
increases  training  costs  per  week  of  training  vhile  it  substantially 
increases  total  cost  by  extending  training  time. 


1 


The  final  section  combines  methodology  and  policy  to  describe  cost 
effective  options  for  classroom  automation.  The  policy  perspective 
induces  fiscal  constraints  which  motivate  cost  effectiveness  concerns  in 
the  first  place.  Cost  effective  options  for  automation  are  described  as 
small  increments  or  add-ons  to  existing  hands-on  software.  Although 
"hands-on  add-ons"  are  not  presently  being  considered  at  the  QMS,  the 
argument  is  made  that  they  may  prove  cost  effective  compared  to  current 
attempts  to  meet  fiscal  constraints  by  cutting  back  on  instructional  staff. 
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AVERAGE  TRAINING  COSTS 


Cost  effectiveness  analysis  for  current  military  training  generally 
starts  with  cost  analysis  because  cost  data  are  typically  more  available 
and  reliable  than  effectiveness  data.  Following  Army  Training  and  Doctrine 
Command  (TRADOC)  policy,  the  Quartermaster  School  (QMS)  at  Fort  Lee, 
Virginia  collects  training  cost  data  on  an  annual  basis.  In  Table  1, 
fiscal  1987  cost  data  for  the  Equipment  Records  and  Parts  Specialist  Course 
(76C10)  are  divided  by  the  total  number  of  76C10  graduates  for  the  same 
year  (3000)  to  obtain  average  training  costs  per  soldier. 

Table  1  employs  the  Cost  Element  Structure  (CES)  Model  (Knapp  and 
Orlansky,  1983).  Before  looking  at  the  data,  certain  characteristics  of 
this  cost  taxonomy  are  worth  noting.  While  the  CES  model  divides  costs 
into  3  time  periods  (i.e.  Research  and  Development,  Initial  Investment,  and 
Operating  and  Support),  this  analysis  of  current  technology  need  not 
include  the  R&D  nor  initial  investment  costs  because  they  were  sunk  many 
years  ago.  Furthermore,  several  CES  sub-categories  for  Operating  and 
Support  do  not  apply  because  the  76C10  Course  presently  uses  very  little 
hardware  and  conducts  very  little  activity  outside  of  the  classroom  (e.g., 
costs  for  petroleum  and  ammunition  are  minimal).  Finally,  no  distinction 
is  made  here  between  military  and  civilian  personnel  costs,  nor  among  a 
large  number  of  indirect  costs  associated  with  running  the  entire  Fort  Lee 
installation.  Cost  data  are  aggregated  across  these  distinctions  because 
by  and  large  they  would  not  be  directly  affected  by  contemplated  changes  in 
technical  instruction,  including  classroom  automation. 


The  Cost  Driver 


Two  main  economic  conclusions  may  be  drawn  from  these  average  cost 
data.  The  first  involves  the  economic  importance  of  the  time  spent  by 
trainees  in  the  classroom  and  thus  in  residence  at  Fort  Lee  (Table  1). 
Trainee  residence  time  directly  determines  their  pay  and  allowances  (40%  of 
total  costs)  and  indirectly  determines  the  share  of  Fort  Lee  "base 
operations"  and  related  support  costs  allocated  to  76C10  training  (37%). 
"Base  operations"  reflects  overhead  expenses  at  Fort  Lee  that  are  charged 
on  the  basis  of  the  number  of  76C10  soldiers  in  residence  (the  "student 
load"),  The  longer  the  Course,  given  a  fixed  graduation  requirement,  the 
greater  ne  number  of  soldiers  in  residence  and  thus  the  larger  the  share 
of  ort  Lee  base  operations  costs  charged  to  76C10  Course. 


1.  Current  methodology  for  military  cost  and  training  effectiveness  analysis 
(CTEA)  provides  an  extensive  accounting  structure  to  identify  and  classify 
training  costs.  The  CES  model  and  the  TCBS  model  (Seidel  and  Vagner,  1977) 
are  both  used  in  this  Report  for  cost  analysis.  The  CES  model  is  used  to 
examine  costs  of  current  76C10  training  because  it  incorporates  training 
process  into  conventional  cost  accounting  in  the  the  Military.  The  TCBS 
model  is  used  later  to  examine  automation  costs  because  it  is  designed 
specifically  to  deal  with  new  training  technology.  For  an  extensive  literature 
review  and  critical  appraisal  of  CTEA  models,  see  Adams,  Goldberg,  and 
Rayhawk  (1986).  For  additionaj.  critical  perspective  see  Soloman  (1986). 


Table  1 


76C10  Training  Costs  Per  Graduate 
During  FY87 


Cost 

Category 

Data  Source 

$  per 
Student* 

Cost 

Share 

A. 

Direct  Instruction 

Supply  Dept. /QMS 

1.  Pay  and  Allowances 

a.  Instructors 

900 

12% 

b.  Supervisors,  Adm,  and 

Supp. 

60 

1% 

2.  Instructional  Materials 

40 

IX 

B. 

Other  Training  Costs 

1.  Battalion 

Supply  Dept. /QMS 

300 

4% 

2.  Directorate  of  Training 
and  Development  (DOTD) 

TRADOC 

300 

4% 

C. 

Soldier/Trainee  Costs 

1.  Pay,  Allowances 

Other 

Supply  Dept. /QMS 

3000 

40% 

2.  PCS  (Travel  to 

Fort  Lee) 

AMCOS  Hodel/ARI 

100 

IX 

D. 

Indirect  Costs 

TRADOC 

1.  Base  Operations 

2400 

32% 

2.  Other  Support 

400 

5% 

Total 

7500 

100% 

*  Actual  numbers  have  been  rounded  off  because  greater  precision  vould 
not  significantly  change  cost  shares. 
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Since  course  length  determines  hov  many  soldiers  must  be  trained  at 
any  one  time  by  resident  staff,  it  also  determines  pay  and  allowances  for 
instructors  and  their  supervisory  and  support  staff  (13X),  battalion 
training  costs  (4%),  and  instructional  materials  cos^s  (IX).  Only  about  5X 
of  training  cost  can  not  be  directly  tied  to  residence  time,  student  travel 
(IX)  and  the  Directorate  of  Training  and  Development  (D0TD)(4X).  But,  out 
of  that,  DOTD  outlays  are  to  some  extent  charged  as  overhead  to  the  Course 
on  the  basis  of  student  load. 

In  summary,  over  95X  of  training  costs  in  this  Course  will  increase  or 
decrease  depending  upon  training  time.  By  itself,  this  economic  reality 
suggests  that  attempts  to  control  or  reduce  training  costs  should  focus 
upon  factors  that  determine  Course  length.  If  indeed  the  primary  purpose 
of  advanced  individual  training  (AIT)  in  76C10  is  to  train  technical 
skills,  then  the  instructional  resources  and  procedures  employed  in  the 
classroom  should  be  a  primary  focus  for  cost  saving  because  they  determine 
the  class  time  necessary  to  learn  technical  skills. 


Leveraging  Inputs  to  Technical  Instruction 

The  average  cost  data  also  suggest  a  second  economic  reason  to  focus 
on  inputs  to  technical  training.  This  reason  involves  the  relatively  low 
share  in  total  training  cost  that  can  be  charged  to  direct  (or  technical) 
instruction  (14X).  From  the  viewpoint  of  potential  upgrades  in  the  design 
and  implementation  of  technical  instruction,  this  small  cost  share 
represents  an  economic  opportunity.  It  means  that  a  relatively  large 
percentage  increase  in  instructional  costs  results  in  a  much  smaller 
increase  in  total  course  costs.  At  the  same  time,  if  improved 
instructional  inputs  reduce  the  time  necessary  to  learn  technical  skills, 
then  the  net  result  may  be  an  overall  cost  savings  as  Course  length 
declines  and  along  with  it  most  other  training  costs. 

In  other  words,  to  the  extent  that  time  requirements  for  technical 
training  determine  Course  length,  the  current  relatively  small  cost  share 
for  technical  instruction  creates  economic  leverage  for  new  inputs  to 
technical  training,  including  new  training  technology.  If  greater  inputs 
or  new  technology  can  make  technical  training  more  productive  in  terms  of 
learning  per  unit  of  time,  then  such  outlays  may  pay  for  themselves.  This 
leverage  will  be  further  explored  below  in  terms  of  dollar  savings  per  hour 
and  per  day  that  training  time  is  reduced,  and  then  in  terms  of  when  during 
the  Course  it  would  be  easiest  to  shorten  training  time. 


The  Limits  of  Current  Methods  for  Classroom  Cost-Effectiveness 

Average  training  cost  per  soldier  gives  a  useful  snapshot  of  the 
Course,  but  such  data  do  not  permit  comparisons  of  costs  and  effects  that 
are  essential  for  cost  effectiveness  analysis.  The  conventional  way  to 
make  such  comparisons  in  training  research  is  to  conduct  experiments.  The 


2.  Shortening  training  time  also  has  the  salutary  effect  of  increasing  the  number 
of  trained  soldiers  on  the  job,  given  any  fixed  Army  budget  for  personnel. 
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current  Course  and  a  nev  training  process  or  technology  vould  be  run  side 
by  side,  all  other  things  being  equal,  and  the  outcomes  and  costs 
compared . 

Unfortunately,  classroom  research  in  the  military  has  had  limited 
usefulness  for  training  cost  effectiveness,  particularily  for  new 
computer-based  technology,  because  of  large  front  end  costs  for  software 
development  and  hardware  procurement,  the  expense  and  disruption  of 
sufficiently  large  controlled  experiments,  and  because  current  test 
measures  typically  show  high  average  levels  of  classroom  achievement  (i.e., 
ceiling  effects).  At  the  same  time,  these  test  data  are  generally  not 
validated  by  subsequent  performance  at  duty  stations  in  the  field.  The 
expense  and  disruption  of  experimental  research  limits  the  number  of 
experiments.  Ceiling  effects  and  the  unproven  reliability  of  outcome 
measures  make  it  difficult  to  justify  the  costs  of  new  technology  on  the 
basis  of  improved  training. 

There  is,  however,  one  important  nev  technology  that  has  had  less 
difficulty  proving  its  worth  :  computer-based  equipment  simulation 
(Orlansky  &  String,  1977,  1981;  Orlansky,  Knapp,  &  String,  1984;  Fletcher  £■ 
Orlansky,  1985).  Hany  flight  and  maintenance  simulators  pay  off  in  the 
economic  sense  that  dollar  measures  of  benefit  exceed  cost.  Such  analysis 
can  be  done  in  these  exceptional  training  circumstances  if  simulation 
constitutes  a  clearly  differentiated  training  activity,  and  if  simulation 
outcomes  can  be  defined  in  terms  of  time  necessary  to  complete  training 
using  actual  equipment.  In  that  case,  costs  can  be  quantitatively  compared 
to  effects  in  terms  of  "transfer-effectiveness  ratios".  Furthermore, 
experimental  research  to  optimize  simulation  use  can  be  gracefully 
incorporated  into  existing  instructional  process  in  terms  of  marginal 
adjustments  in  the  amount  of  time  students  spend  on  simulators  or  marginal 
changes  in  simulation  technology. 

It  is  the  working  hypothesis  of  this  Report  that  automation  of 
classroom  training  can  extend  the  domain  for  CTEA  analysis  in  a  way  similar 
to  equipment  simulation.  It  does  so  by  helping  to  differentiate  training 
activities  into  technically  and  economically  distinct  subprocesses. 
Automated  activities  are  separately  programed;  computer  equipment  is 
separately  budgeted;  and,  in  order  to  evaluate  nev  applications,  the  mix  of 
instructor-  and  computer-initiated  activities  can  be  adjusted  along 
existing  seams  that  normally  connect  different  classroom  activities.  These 
changes  wrought  by  computers  invite  quantitative  analysis  and  cost 
effectiveness  tradeoffs. 

However,  compared  to  the  equipment  simulators  mentioned  above, 
classroom  computers  are  much  less  self  contained.  Their  instructional 
value  derives  much  more  from  functional  integration  with  all  other  forms  of 
classroom  instruction.  This  complexity,  coupled  vith  the  apparent  success 
of  current  instruction,  make  it  difficult  to  justify  automation  on  the 
basis  of  greater  effectiveness.  Vith  the  exception  of  job  task  simulation 
(See  policy  section  belov),  the  cost  effectiveness  of  classroom  automation 
is  most  likely  based  on  cost  savings. 
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Furthermore,  eutomation  is  more  likely  to  prove  cost  effective  as  a 
series  of  marginal  Increments,  each  one  Integrated  into  classroom  process 
before  attempting  the  next,  because  large  changes  can  easily  prove  too 
disruptive  and  too  difficult  for  existing  outcome  measures  to  evaluate. 
These  practical  considerations  are  reinforced  by  methodology  and  policy 
described  belov.  The  next  Section  of  this  Report  describes  current 
classroom  training  as  a  productive  process,  where  margins  for  cost  saving 
automation  can  be  identified,  if  they  exist. 


CLASSROOM  TRAINING  AS  PRODUCTION  PROCESS 

The  76C10  Course  (and  many  other  AIT  courses  in  the  Army)  is  diviled 
into  annexes,  and  each  annex  is  broken  down  into  separate  blocks  of 
instruction.  Instruction  blocks  vary.in  length,  but  the  pattern  of 
activity  in  each  is  roughly  the  same.  First  comes  platform  lecture,  for  a 
prescribed  period  of  time,  in  order  to  present  new  material  and  to  answer 
questions  from  the  entire  class.  Averaged  over  the  entire  Course,  about 
502  of  class  time  is  prescribed  for  platform  presentation,  although  not  all 
of  this  time  is  actually  devoted  to  technical  training  (Figure  1). 
Frequently,  instructors  interrupt  technical  training  in  order  indoctrinate 
or  "soldierize"  trainees,  who  by  and  large  are  relatively  new  to  Army 
organization  and  discipline.  In  fact,  the  dual  role  of  platform 
instructors,  as  authorities  in  both  job  skills  and  other  military  doctrine, 
restricts  opportunities  for  classroom  automation  because  computers  are  much 
less  effective  for  indoctrination. 

Following  lecture,  all  soldiers  receive  a  printed  set  of  instructions 
and  questions  as  a  practical  exercise  (PE).  Since  PE  scores  are  generally 
not  recorded,  this  is  mainly  a  time  for  soldiers  to  check  themselves  and  to 
seek  more  personal  attention  from  instructors,  if  needed.  Instructors  may 
also  monitor  student  performance  at  this  time  in  order  to  select  slower 
students  to  attend  study  hall  after  normal  classroom  hours.  PEs  take  up 
about  402  of  class  time. 

Test  and  evaluation  of  soldier  knowledge  and  skills  occurs  at  the  end 
of  an  instructional  annex.  An  annex  is  made  up  from  5-17  blocks.  For 
students  who  fail  the  initial  end-of-annex  test,  remedial  training  is 
required  during  off-hours,  and  then  they  are  immediately  retested.  Failure 
to  pass  a  second  time  results  in  recycling  back  to  take  the  annex  over 
again.  (Repeated  recycling  results  in  being  dropped  from  the  course.) 
Testing  and  related  reviews  and  critiques  take  up  about  102  of  class  time. 


3.  All  instruction  in  this  Course  pertains  to  prescribed  procedures  for  taking 
inventory,  filling  out  request  and  file  forms,  using  appropriate  manuals 
as  references,  and  filing  documents  properly. 
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In  the  official  Program  of  Instruction  (POI)  (1987),  this  entire 
instructional  sequence  is  defined  or  denominated  in  terms  of  instructor 
contact  hours  (ICH).  Each  stage  of  the  process  shall  consist  of  a  certain 
number  of  ICH.  The  entire  Course  requires  360  ICH.  Because  there  are 
only  36  classroom  hours  in  a  5-day  training  veek,  the  360  ICH  requirement 
translates  into  a  10-11  veek  residence  time  at  Port  Lee  for  trainees 
(allowance  must  be  made  for  programmed  days  off  plus  holidays). 

During  the  two  main  instructional  activities  (lecture  and  PE),  it  is 
expected  that  most  learning  will  take  place  under  the  direction  of 
instructors.  All  soldier  trainees  are  supposed  to  learn  at  the  same  time, 
in  "lock-step",  following  directions  from  the  platform.  Instructors 
provide  personal  attention  to  individual  students,  but  their  limited 
training  in  instructional  methods  and  the  procedural  nature  of  the  subject 
tend  to  restrict  instructional  process  to  repetitive  presentation  of  the 
same  material,  followed  by  drill  and  practice  routines.  Furthermore,  peer 
interactions  among  students  in  the  classroom  is  generally  discouraged  (with 
the  exception  of  cooperative  learning,  see  below),  and  few  opportunities 
exist  for  student  initiative. 


Figure  1 


The  Organization  of  Instructor  Contact  Hours  (ICH) 
During  a  Generic  Instruction  Block 


50*  40*  10* 

Lecture  Practical  Exercise  (PE)  Testing 


Source:  76C10  Master  Scheduling  Form,  Fall,  1986. 


This  brief  overview  of  76C10  training  process  sets  the  stage  for  a 
quantitative  analysis  of  its  cost  and  effectiveness.  Vith  training  as 
currently  organized  into  a  "lockstep"  procedure,  the  notion  of  training 
time  as  cost  driver  can  be  further  explicated  in  terms  of  instructor 
contact  hours  (ICH).  Hov  does  instructor  contact  vith  a  class  constitute  a 
flow  of  costly  inputs  that  may  be  charged  to  76C10  training?  On  the  other 
hand,  hov  does  instructor  contact  generate  a  flow  of  training  effects  or 
products  that  justify  the  cost  of  expensive  inputs? 

Both  questions  are  addressed  below  in  terms  of  cost  and  product  curves 
that  describe  cost  and  product  flows,  hour  by  hour.  These  "marginal"  and 
"average"  curves  are  basic  in  the  economic  theory  of  production.  Economics 
is  a  science  for  making  comparisons  among  alternative  options  and  outcomes; 
and  these  economic  curves  will  be  used  to  suggest  hov  adjustments  in  the 
training  schedule  and  perhaps  investment  computer-based  instruction  may 
increase  productivity  per  unit  time  and  thus  payoff  in  terms  of  reduced 
total  training  time. 
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Costs  will  be  discussed  first,  as  a  function  of  the  flow  of  ICH, 
because  this  relationship  involves  only  straightforward  accounting,  using 
data  already  collected  for  administrative  purposes.  Productivity  curves, 
considered  second,  require  closer  examination  of  how  or  when  instruction 
results  in  learning.  In  both  cases,  hovever,  notice  that  the  analysis 
pertains  first  to  a  class  of  40  students.  Classroom  data  are  then  used  to 
compute  cost  per  student.  Also  notice,  for  both  cost  and  productivity 
analysis,  instruction  time  (as  calibrated  into  ICH)  serves  as  the  critical 
explanatory  variable. 


Classroom  Cost 


The  following  cost  analysis  takes  advantage  of  the  fact  that  in 
lock-step  training,  the  causal  relationship  between  the  flow  of  training 
inputs  and  the  flow  of  outputs  goes  in  only  one  direction.  With  the  minor 
exception  of  study  halls,  the  flow  of  costly  inputs  does  not  vary  as  a 
function  of  when  learning  takes  place.  This  simplification  makes  it 
possible  in  the  first  instance  to  consider  costs  independently  of  effects 
or  productivity.  Later  on,  costs  and  product  will  be  combined  as  they 
must  in  order  to  evaluate  cost  effectiveness. 

Furthermore,  lock-step  training  policy  makes  it  unnecessary  to 
differentiate  among  the  cost  components  (see  Table  1  and  related 
discussion),  because  the  input  mix  does  not  change  during  the  course  of 
instruction.  All  soldiers  remain  in  the  classroom,  along  with  2-3 
instructors,  at  all  times  (or  at  least  without  regard  to  the  schedule  of 
instructional  activities).  In  other  words,  instructor  contact  hours  (ICH) 
is  a  convenient  surrogate  for  all  factor  inputs. 

During  a  block  of  instruction,  the  same  training  cost  is  incurred, 
hour  by  hour,  because  the  process  involves  a  constant  set  of  costly  inputs. 
As  shown  by  the  lower  line  in  Figure  2,  marginal  cost  (MC)  per  hour  of 
instruction  can  be  illustrated  as  a  straight,  flat  line.  Notice  that  the 
horizontal  axis  is  a  slight  truncation  of  the  ICH  scale  presented  in  Figure 
1  (testing  is  omitted).  The  vertical  scale  represents  dollars.  Since  it 
costs  approximately  $300k  to  train  a  class  of  40  soldiers  for  360  ICH,  the 
marginal  cost  per  ICH  equals  about  $830.  This  might  simply  be  called  the 
cost  per  ICH,  because  it  does  not  vary  over  time  within  an  instruction 
block  or  among  instruction  blocks,  but  a  marginal  aspect  still  exists  in 
the  sense  that  training  inputs  and  thus  costs  could  differ  during  different 
parts  of  the  Course  depending  upon  the  instructional  goal  and  the  subject 
matter  (See  below).  At  the  moment,  however,  constant  costs  per  ICH  are  the 
rule. 


4.  Costs  would  not  be  independent  of  effects  if  training  were  self-paced  as  it 
was  prior  to  1983.  In  that  earlier  training  program,  improved  instruction 
lowered  costs  directly  by  shortening  training  time.  In  lockstep  training, 
the  interplay  between  cost  and  effectiveness  involves  indirect,  administrative 
actions. 
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Figure  2 

Cora  Over  Time  During  •  Generic  Instruction  Block 


Lecture  PE 

Time  Measured  in  Instructor  Contact  Hours  (ICH) 


10 


The  upper  line  in  Figure  2  illustrates  total  training  costs  per 
instruction  block.  Total  cost  equals  the  sum  of  marginal  costs  or  the  area 
under  the  marginal  cost  curve.  The  level  of  marginal  cost  curve  is  the 
slope  of  the  total  cost  curve.  Note  that  total  costs  are  left  indefinite 
since  they  depend  upon  the  variable  number  of  hours  allocated  to  each 
block. 

Bovever,  the  level  marginal  cost  curve  permits  calculation  of  cost 
savings  that  may  obtain  from  shortening  training  time.  One  instructor 
contact  hour  (ICH)  costs  $830,  and  if  95%  of  these  costs  could  be  saved  by 
shortening  training  time,  then  an  hour  reduction  for  one  class  saves  $790. 
Over  a  year,  an  hour  reduction  for  75  classes  would  save  $59,000.  Saving  a 
day  of  instruction  (7.2  hours),  for  all  classes  trained  for  one  year,  would 
save  almost  $425,000  in  annual  training  costs.  Actual  cost  savings  would 
require  more  detailed  cost  accounting,  but  this  ballpark  estimate  will  be 
used  later  on  to  explain  how  investment  in  computer  software  may  leverage 
offsetting  cost  savings  by  shortening  the  length  of  training.  The 
following  discussion  of  training  productivity  is  designed  to  identify  when 
time  savings  are  most  easily  achieved  during  the  course  of  training. 

Classroom  Productivity 

Total  output  from  the  Course  can  be  measured  in  annual  terms 
comparable  to  annual  cost  data.  About  3000  76C10  soldiers  graduated  in  FY 
1987.  This  total  can  also  be  disaggregated  into  shorter  time  intervals, 
but  unlike  marginal  costs,  there  is  no  accounting  procedure  to  derive 
marginal  (hour  by  hour)  classroom  productivity.  While  policy  and 
established  administrative  procedures  can  prescribe  the  incidence  of 
training  costs,  they  can  not  prescribe  when  students  learn  the  required 
material.  In  order  to  develop  a  marginal  productivity  curve,  the  product 
of  training  must  first  be  defined  in  practical  terms  that  permit  students 
to  demonstrate  knowledge  and  skill. 

Productivity  in  the  76C10  Course  is  measured  almost  entirely  by 
objective  tests,  administered  at  the  end  of  each  annex  as  well  as  at  the 
end  of  the  entire  course.  The  success  criterion  is  a  single  cut  score 
(e.g.  85%)  and  little  benefit  accrues  to  either  the  school,  the  instructor, 
or  the  student  for  test  scores  that  exceed  this  threshold.  A  similar 
performance  criterion  is  used  at  many  Army  schools  for  advanced  individual 
training  (AIT)  (perhaps  because  they  all  share  the  same  difficulty  in 
trying  to  validate  test  instruments  against  performance  in  the  field). 

Accepting  this  pragmatic  performance  criterion,  marginal  classroom 
productivity  may  be  defined  as  the  number  of  soldiers  meeting  the  test 
criterion,  hour  by  hour  during  an  instruction  block.  Figure  3  illustrates 
such  a  curve  as  a  relationship,  driven  in  real  time,  by  instructor  contact 
hours  (ICH).  The  lover  curve  (with  the  inverted  nU"  shape)  is  the  marginal 
product  (HP)  curve.  The  upper  curve  (with  approximate  "S"  shape)  is  the 
total  product  (TP)  curve  or  the  cumulative  sum  of  soldiers  meeting 
criterion  (i.e.  the  area  under  the  HP  curve). 
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%  of  Class  Achieving 
Criterion  Performance 


Figure  3 

Productivity  Over  Time  During  a  Generic  Instruction  Block 
(illustrative  Hypothetical  Data) 
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Unfortunately,  since  existing  tests  are  administered  only  at  the  end 
of  each  annex  and  at  the  end  of  the  course,  existing  test  data  do  not 
permit  estimation  of  hour  by  hour  or  even  block-specific  productivity 
curves.  Formal  estimation  of  the  latter  would  involve  sampling  out  of  the 
class,  during  the  course  of  instruction.  More  informal  estimation 
procedures  could  also  be  used,  such  as  self  evaluation  by  students  based 
upon  when  they  can  correctly  answer  test  questions  that  would  be  passed  out 
at  the  beginning  of  instruction.  The  objective  of  this  analysis  is  to  show 
why  such  data  gathering,  by  whatever  means,  may  be  worth  the  effort.  Prior 
to  collection  of  data,  but  based  upon  extensive  classroom  observations,  the 
curves  in  Figure  3  illustrate  a  plausible  hypothesis  about  productivity  in 
the  76C10  Course. 

Since  classroom  marginal  productivity  is  not  a  familiar  notion,  the 
basic  concept  may  be  explained  first  by  an  analogy  to  popping  popcorn  over 
a  constant  heat  source.  Beat  is  applied  and  at  first  nothing  happens. 

Then  one  kernel  pops;  then  another  and  another  as  the  rate  increases  until 
most  kernels  pop  in  a  flurry.  Then  the  popping  tapers  off,  showly,  until 
only  a  few  unpopped  kernels  remain.  Current  testing  in  76C10  Course,  at 
the  end  of  each  annex,  is  analogous  to  measuring  the  total  number  of  popped 
kernels  emptied  from  the  container  after  removing  it  from  the  heat. 

The  marginal  productivity  curve  (Figure  3)  shows  when  individual 
students  "pop  above"  the  performance  criterion.  However,  before  examining 
the  shape  of  this  curve  for  76C10  training,  it  should  be  noted  that 
productivity  could  be  measured  in  different  ways,  with  different  curves 
resulting.  For  example,  in  making  popcorn,  the  popping  process  might  have 
been  more  sensitively  monitored  in  terms  of  the  number  of  kernels  heating 
up  times  the  average  rate  at  which  they  heat  up.  Presumeably,  this  curve 
would  start  out  on  a  peak  or  high  plateau,  when  all  kernels  would  start  to 
warm  up  from  room  temperature,  and  then  taper  off  first  because  the  heat 
rate  falls  as  the  kernels'  temperature  rises  and  then  because,  as  kernels 
pop,  fewer  kernels  would  be  left  to  heat.  Although  the  rigorous  case  has 
not  been  made,  it  should  be  clear  that  this  more  sensitive  measure  of 
productivity  would  show  more  production  during  the  early  phase  of  popping, 
compared  to  simply  measuring  when  kernels  pop. 

A  more  sensitive  measure  of  productivity  in  training  might  be 
considered  if  rate  of  learning  could  be  estimated.  However,  the  latter  is 
much  more  difficult  to  define  than  in  the  simple  case  of  popcorn,  and 
certainly  more  expensive  to  measure  than  simply  noting  when  students  "pop 
above"  the  performance  criterion.  Furthermore,  the  same  argument  can  be 
made  (as  in  the  case  of  popcorn)  that  the  more  sensitive  measure  would  show 
greater  production  during  the  early  phase  of  training,  and  less  production 
later  on.  Since  the  difference  between  learning,  early  and  late,  lies  at 
the  crux  of  the  argument  presented  below,  the  use  of  the  less  sensitive 
measure  of  productivity  implies  that  a  better  measure  would  only  reinforce 
the  conclusions. 
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Assuming  the  performance  criterion  measure  as  it  is  used  in  the  76C10 
classroom,  the  approximate  position  and  shape  of  the  marginal  product  and 
total  product  curves  can  be  postulated  on  the  basis  of  the  instructional 
process  and  the  mix  of  soldiers  in  the  classroom.  Soon  after  instruction 
begins,  this  curve  can  be  expected  to  rise  from  zero,  gradually  at  first 
and  then  more  and  more  rapidly  until  the  lecturer  has  explained  all  new 
procedures.  The  initial  rise  reflects  the  fact  that  some  students  have 
previous  experience  with  the  material,  or  are  exceptionally  adept  or 
self-motivated,  and  thus  will  more  or  less  teach  themselves  quickly,  if 
necessary  using  directions  contained  in  printed  manuals. 

Most  trainees,  however,  will  wait  for  the  instructor  to  cover  the 
required  material  because  there  is  little  institutionalized  incentive  to 
get  ahead  or  to  seek  understanding  of  details  greater  than  is  required  by 
the  instructor.  Immediately  after  all  new  material  has  been  presented, 
perhaps  50X  of  the  class  can  meet  the  proficiency  criterion.  As  the 
lecture  continues  on,  fewer  and  fewer  new  students  meet  criterion,  because 
a  majority  of  students  already  have  done  so,  because  further  instruction  is 
generally  limited  to  repeated  review  of  procedural  details  (i.e.  not 
tailored  to  slower  students),  and  because  of  the  large  apparent  variability 
in  aptitude  or  motivation  (despite  pre-screening)  among  those  who  learn 
slowly.  Since  the  course  length  appears  to  be  determined  by  the 
requirements  of  slower  students,  the  marginal  productivity  curve  can  be 
expected  to  have  a  relatively  long  right  tail,  running  near  the  horizontal 
axis . 


Based  upon  extensive  classroom  observations,  the  MP  curve  in  Figure  3 
also  illustrates  a  plausible  range  of  variation  between  the  "modal"  or  peak 
number  of  students  (25X  of  the  class)  who  reach  criterion  Just  after  the 
initial  presentation  of  new  material,  and  the  low  number  (5X)  reaching 
criterion  near  the  end  of  the  right  tail  of  the  curve.  In  the  following 
analysis,  the  shape  of  the  marginal  productivity  curve  and  the  pattern  of 
shapes  among  instruction  blocks  determine  opportunities  to  improve  training 
cost  effectiveness. 


Marginal  Cost  per  Student 

These  two  pairs  of  curves  that  describe  a  classroom  of  students,  one 
for  c..'St  and  the  other  for  product,  can  now  be  combined  to  calculate 
training  costs  per  student,  as  a  function  of  instructor  contact.  Cost  per 
graduate  can  be  obtained  by  aggregating  or  adding  costs  across  all 
instruction  blocks  and  annexes.  For  simplicity,  however,  we  will  continue 
to  consider  only  one  instruction  block,  assuming  it  is  representative  of 
the  entire  course.  If  this  were  true,  then  the  variability  in  average  and 
marginal  costs  within  this  representative  instruction  block  would  reflect 
such  variation  over  the  entire  Course.  On  the  other  hand,  differences 
among  instruction  blocks  will  prove  critical  in  targeting  the  application 
of  computer  technology. 


« 
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The  three  frames  in  Figure  4  illustrate  the  mathematics  involved.  All 
three  frames  line  up  along  the  same  horizontal  axis.  The  top  two  frames  [a 
&  b]  merely  reproduce  Figures  2  and  3  respectively.  The  bottom  frame  [c] 
takes  the  ratio  of  cost  per  class  from  [a]  and  product  per  class  from  [b] 
to  obtain  cost  per  percentile  of  the  class.  The  latter  can  easily  be 
converted  to  cost  per  student.  The  marginal  cost  per  percentile  or  per 
student  (i.e.,  the  more  irregular,  "u"  shaped  curve)  equals  classroom 
marginal  cost  divided  by  classroom  marginal  product.  The  average  cost  per 
student  equals  classroom  total  cost  divided  by  classroom  total  product. 
Notice  that  the  marginal  cost  curve  Intersects  the  average  cost  curve  at 
its  minimum  point  (marked  "M"). 

Both  the  average  and  marginal  cost  curves,  if  aggregated  across  all 
blocks  and  annexes  in  the  Course,  invite  comparison  to  the  single  average 
cost  per  soldier  for  the  entire  Course  (Table  1).  However,  the  marginal 
cost  curve  highlights  the  effective  range  of  cost  variation  among 
individual  soldiers.  If  the  peak  of  the  marginal  classroom  productivity 
curve  accounts  for  10  soldiers  (25£  of  the  class  of  40  students),  and  the 
low  point  on  the  right  tall  accounts  for  only  2  students  (5X),  then 
training  cost  per  soldier  in  the  small  group  is  5  times  greater  than  for 
each  soldier  in  the  large  group.  Without  actual  data  for  classroom 
productivity,  this  difference  remains  hypothetical,  but  large  differences 
are  at  least  possible,  if  not  probable.  In  any  case,  for  Course  managers 
and  higher  level  administrators,  this  description  of  costs  as  a  curve, 
instead  of  a  single  number,  helps  to  identify  cost  effectiveness  tradeoffs. 

A  basic  tradeoff  compares  cost  savings  from  shortening  programmed 
training  time  to  cost  increments  that  may  result  if  shortened  programs 
result  in  more  recycles.  Shortening  programmed  training  time  reduces  costs 
for  all  soldiers  reaching  criterion  within  the  shorter  period,  but  it 
increases  training  costs  for  those  who  must  repeat  failed  annexes.  Note 
that  actual  Course  length  (or  required  training  time  averaged  over  all 
graduates)  exceeds  programmed  Course  length  by  an  additive  factor  that  is 
proportional  to  the  number  of  recycles.  With  actual  productivity  data,  the 
tradeoff  could  be  systematic. 

Expected  variation  in  marginal  productivity,  among  blocks,  implies 
that  programmed  training  time  can  be  shortened  more  easily  in  certain 
blocks  than  in  others.  Figure  5  illustrates  two  extreme  cases  for 
classroom  marginal  productivity.  Frame  [a]  shows  a  relatively  large  window 
of  opportunity  to  reduce  costs  by  shortening  the  Course.  This  MP  curve  has 
a  short  upward  sweep  and  a  right  tail  that  is  long  and  low.  Reducing 
classroom  hours  here  would  adversely  affect  fewer  soldiers  than  in  Frame 
[b],  where  the  HP  curve  has  a  long  upward  sweep  and  a  short,  steeply  sloped 
right  tail.  If  actual  classroom  productivity  data  were  collected,  then  an 
obvious  way  to  reduce  training  time  at  zero  net  cost  would  be  to  adjust 
programmed  training  time  in  all  instructional  blocks  so  that  an  hour  time 
reduction,  allocated  to  any  block,  would  have  the  same  net  effect  on  cost 
due  to  increased  recycles. 
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Similarly,  the  instructional  block  described  in  Prame  [ a ]  offers 
greater  opportunity  to  shorten  the  course  by  upgrading  instruction. 

Along  the  long,  low  right  tail  of  the  MP  cjrve,  most  soldiers  have  already 
mastered  the  required  skills  and  only  a  fev  have  not.  As  discussed  further 
belov,  this  is  a  likely  situation  to  engage  the  larger  group  on  computers 
so  that  instructors  can  focus  greater  attention  on  the  smaller  group. 

Frame  (a]  offers  greater  opportunity  to  shorten  the  course  than  Prame  [b] 
because  there  are  fever  slower  learners  to  speed  up,  per  hour  saved. 

Average  training  time  may  be  shortened  by  a  variety  of  other  measures, 
such  as  allowing  soldiers  to  move  ahead  once  they  have  met  criterion 
performance,  or  engaging  faster  students  to  assist  the  others  (See 
discussion  of  cooperative  learning  belov).  However,  the  main  purpose  here 
is  to  consider  automation  options.  Computers  can  be  j^sed  in  different  ways 
to  train,  and  some  options  save  time  and  some  do  not.  One  option, 
automation  of  hands-on  practical  exercises,  has  already  been  implemented  in 
a  3-week  extension  of  the  basic  76C10  Course  (See  TACCS/SAMS  belov),  and 
the  same  option  has  been  approved  for  implementation  in  76C10  during  the 
next  fiscal  year.  A  reviev  of  current  automation  policy  at  this  point  will 
help  to  determine  which  cost, and  productivity  data  and  which  automation 
options  are  policy  relevant. 


5.  For  a  review  of  cost  effectiveness  data  for  computer-based  instruction, 
as  a  substitute  for  conventional  classroom  training,  see  Orlansky  and 
String  (1979).  These  data  were  collected  prior  to  the  advent  of 
inexpensive  microcomputers,  but  favorable  implications  for  the  cost 
effectiveness  of  individualized  instruction  are  only  reinforced  by  recent 
technological  developments. 

6.  For  an  early  discussion  of  the  need  to  define  automation  options 
realistically,  see  Seidel  and  Kopstein  (1970). 
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CURRENT  SCHOOL  POLICY  FOR  CLASSROOM  AUTOMATION 


The  Hands-On  Training  Requirement 

The  76C10  Course  is  not  automated,  but  there  Is  little  doubt  that  it 
will  be.  Not  automating  is  not  an  option.  Computers  vill  be  used  in  the 
classroom  because  they  are  or  soon  vill  be  used  as  tools  on  the  job. 
Computers  vill  be  used  to  simulate  automated  job  tasks  because  "hands-on” 
the  actual  equipment  is  the  best  vay  to  train  inside  or  outside  of 
classrooms.  The  Department  of  the  Army  supports  this  policy  by  supplying 
automated  vorkstations  to  the  Quartermaster  School,  at  no  charge,  just  as 
it  supplies  vorkstations  to  operational  units. 

This  automation  requirement  is  policy  background  for  all  questions 
about  the  cost  effectiveness  of  CBT.  The  issue  is  not  vhether  76C10  vill 
be  automated,  but  hov  the  necessary  introduction  of  computers  into 
classrooms  should  be  managed.  Alternatively,  the  issue  is  not  vhether 
automatiog  is  cost  effective;  but  only  hov  to  make  it  as  cost  effective  as 
possible. 

The  first  hands-on  automation  planned  for  76C10  involves  the  Unit 
Level  Computer  (ULC)  and  the  Unit  Level  Logistics  System  (ULLS).  Training 
on  both  is  currently  planned  to  extend  the  Course  by  65  hours  or  about  9 
days  (a  17X  increase).  The  ULC  computer  is  a  stand  alone,  general  purpose 
microcomputer  that  according  to  design  specifications  vill  have  systems 
softvare  amenable  to  training  as  veil  as  to  field  operations.  (Actual 
equipment  has  not  yet  been  procured  by  the  Army.)  The  Supply  Department  at 
the  Quartermaster  School  is  scheduled  to  receive  176  ULC  machines  during 
the  third  quarter  of  FY88.  132  machines  vill  be  used  to  equip  3 
classrooms,  the  number  necessary  to  cover  a  tvo  veek  interval,  since  2 
classes  of  up  to  44  students  start  training  every  other  veek,  vith  one 
class  starting  during  the  veek  in-betveen.  Training  on  the  ULC^ULLS  system 
is  planned  to  begin  during  the  first  or  second  quarter  of  FY89. 


7.  This  conclusion  is  based  upon  extensive  conversations  vith  instructors  and 
managers  in  the  76C10  course,  vith  course  developers  and  managers  vithin  the 
Supply  Department  and  the  Directorate  of  Training  Development 

(DOTD),  and  it  is  based  upon  the  1987  QMS  Individual  Training  Plan  for  76C10. 
Furthermore,  discusssion  at  the  1987  National  Security  and  Industrial 
Association  Workshop  on  Manpover  and  Training  indicated  that 
hands-on  training  is  of  highest  priority  for  advanced  individual  training 
(AIT)  at  many  Army  schools  and  in  the  other  Services  as  veil. 

8.  It  is  expected  that  some  EIDS  (Electronic  Information  Delivery  System) 
vorkstations  vill  also  be  available  for  use  in  76C10.  Hovever  current  plans 
vould  employ  this  equipment  only  for  ongoing  research  into  computer-based 
instruction. 

9.  Procurement  and  implementation  plans  for  ULC/ULLS  obtained  from  Mr. 

Lev  Thayer,  Branch  Chief  in  charge  of  76C10  training  at  the  QMS. 

19 


Many  details  related  to  the  plan  of  instruction  and  costs  have  yet  to 
be  worked  out,  but  discussion  with  Course  manager s  indicates  that 
implementation  will  generally  follow  the  precedent  established  by  a  current 
extension  of  the  basic  76C10  Course,  the  3-week  Course  designed  to  teach 
the  Tactical  Army  Combat  Support  Computer  System/Standard  Army  Maintenance 
System  or  TACCS/SAMS. 


Computer  Use  in  the  TACCS/SAMS  Extension 

Automation  policy  at  the  Quartermaster  School  has  been  demonstrated 
already  in  terms  of  an  extension  of  training  for  a  subset  of  76C10 
graduates.  For  about  two  years,  the  Supply  Department  at  Fort  Lee  has  held 
over  about  10X  of  76C10  graduates  for  3  weeks  in  order  to  train  them  on 
TACCS/SAMS  (Tactical  Army  Combat  Support  Computer  System/Standard  Army 
Maintenance  System).  While  most  76C10  graduates  work  at  the  lower  "Unit" 
level,  TACCS/SAMS  is  used  at  the  higher  organizational  level  of  "Direct 
Support”  (DS).  TACCS/SAMS  is  a  new  automated  system  for  DS  operations. 
Because  the  basic  76C10  training  deals  only  with  manual  DS  procedures, 
TACCS/SAMS  requires  a  3-veek  extension  of  the  basic  course. 

In  this  first  application,  the  only  purpose  for  computers  is  to  give 
students  initial  hands-on  simulation  of  automated  job  tasks  that  is  as 
realistic  as  possible.  Computer  use  is  limited  entirely  to  practical 
exercises  where  the  nature  of  automated  training  is  predetermined  by 
computer  use  on  the  job.  Other  than  the  substitution  of  computers  for 
paper  and  pencil  during  these  practical  exercises,  the  basic  instructional 
design  remains  unchanged.  Soldiers  follow  the  platform  instructor  in 
lock-step,  during  the  presentation  of  all  new  material.  Bence,  there  is 
little  need  for  course  developers  to  consider  the  instructional 
capabilities  of  these  machines  beyond  what  would  be  done  by  subject  matter 
experts,  in  their  usual  roles  as  course  developers  and  instructors. 

Nevertheless,  this  introduction  of  computers  into  the  classroom 
involves  costs.  The  Training  Cost  Breakdown  Structure  (TCBS)  (Seidel  and 
Vagner,  1977)  has  been  designed  to  examine  costs  when  new  automation 
technology  is  introduced  into  a  training  system.  While  the  Cost  Element 
Structure  (CES)  (Knapp  and  Orlansky,  1983)  could  be  used  for  this  purpose, 
its  fidelity  to  military  budgeting  is  less  important  than  the  functional 
classif ication  scheme  used  in  TCBS.  For  example,  CES  repeatedly 
distinguishes  between  military  and  non-military  personnel  costs  while  TCBS 
makes  no  such  distinction  at  all.  If  the  QMS  were  contemplating  automation 
as  a  major  overhaul  of  76C10,  then  it  would  be  more  appropriate  to  use 
military  accounting  conventions.  But  as  long  as  the  focus  is  primarily  on 
complements  to  the  present  lockstep  system  (based  upon  instructor 
initiative),  and  as  long  as  experience  with  classroom  automation  remains 
limited,  the  TCBS  identification  of  costs  of  computer  based  training  (CBT) 
by  system  component  (e.g.,  terminals,  utility  programs,  and  memory) 
involves  fewer  assumptions. 
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Furthermore,  TCBS  is  a  more  workable  accounting  system  for  planning 
purposes  since  costs  of  automation  are  changing  rapidly,  driven  by 
industry-wide  trends  in  technology  that  are  specific  to  CBT  components. 

Even  after  component  technology  and  costs  settle  down,  it  takes  time  for 
course  administrators  and  cost  accountants  to  fit  the  relatively  new 
training  costs  (e.g.,  new  classrooms  designed  for  computer  workstations, 
machine  maintenance)  into  existing  programs  and  budgets.  Hence,  even 
though  TACCS  machines  have  been  used  in  a  classroom  for  about  a  year,  their 
impact  on  lES-type  administrative  budgets  remains  less  clear  than  in 
technologically  specific  TCBS  accounts. 

Table  2  employs  TCBS  accounting  to  describe  automation  costs  for  the 
hands-on  PE  segment  of  TACCS/SAMS  training.  Note  that  certain  CES 
accounting  items,  such  as  AIT  soldier  pay  and  allowances  or  "base  ops,"  are 
excluded  because  they  would  not  be  affected  by  use  of  computers  for 
hands-on  practical  exercises.  Of  course,  the  basic  rationale  for  this 
Course  extension  is  the  need  to  teach  soldiers  how  to  use  an  automated 
system  in  the  field,  and  in  that  sense  all  training  costs  could  be 
considered  "automation  costs."  But  our  concern  here  is  only  to  evaluate 
training  costs  and  effects  with  and  without  computers  in  the  classroom. 
Presumably,  soldiers  could  be  trained  on  TACCS/SAMS  without  actually 
touching  a  computer,  just  as  the  present  76C10  Course  trains  automated 
prescribed  load  list  (PLL)  procedures  using  pencil  and  paper. 

Table  2  shows  that  TACCS/SAMS  hands-on  automation  involved  no  QMS 
investment  for  the  development  and  procurement  of  operating  workstations. 
Hardware  costs  the  School  nothing  because  the  QMS  received  computers  as  part 
of  the  Army-wide  procurement  that  placed  TACCS  machines  in  direct  support 
(DS)  units  in  the  field.  Similarly  for  computer  systems  software,  because 
the  QMS  does  not  use  their  TACCS  machines  for  any  other  instructional 
purpose  than  simply  emulating  field  operations.  Finally,  no  costs  were 
incurred  specifically  to  develop  a  new  instructional  system  since  the 
TACCS/SAMS  Course  extension  conforms  to  the  76C10  mold.  In  summary,  the 
QMS  automated  this  first  segment  of  76C10  with  minimal  front  end  investment 
that  could  be  charged  specifically  to  the  automation  of  training. 

Of  course,  new  instructional  materials  were  developed  (line  5),  but 
this  was  done  over  a  short  period  of  6  months  and  a  similar  investment 
would  have  been  required  to  teach  this  new  course,  with  or  without 
automated  hands-on.  Similarly,  new  facilities  would  have  to  be  built  or 
found,  with  or  without.  However,  automation  added  significant  new 
facilitites  costs  as  it  increased  requirements  for  space,  space 
conditioning,  furniture,  and  electrical  utilities. 
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Table  2 


Computer-Related  Outlays  and  Costs  for  TACCS/SAMS  Extension  of  76C10 


1.  Hardware*  $0  (QMS)** 

$220,000  (Dept,  of  Army) 

(11  TACCS-V2  workstations  at  $20k  per,  1984  procurement  plus 
upgrades) 

2.  Facilities*  $72,000  (QMS) 

(one  classroom  renovated  plus  new  furniture) 


3.  Computer  systems  software*  $0  (QMS) 

$?  (Dept  of  Army) 


4.  Instructional  system  development  $0 

and  instructor  preparation 


5.  Instructional  materials*  $58,000  (QMS) 


6.  Operation  and  maintenance 

consumables  $600/yr/machine 

or  $6600/yr  total 

machine  maintenance  $1400/machine/yr 

or  15,400/yr  total 


instructor  pay  and  allowances  $125. 500/year 

[(1)  E7  and  (3)  E6's] 


Source:  Supply  Department  at  the  Quartermaster  School,  Fort  Lee,  VA. 

*  Initial  Invest  outlays,  not  annual  costs 

**  Parentheses  adjacent  to  cost  estimate  indicates  who  pays. 


Note:  Although  TCBS  accounting  partitions  costs  into  3  phases  of  an  automated 
system's  lifecycle,  all  3  phases  (development,  procurement,  and 
operation/maintenance)  can  be  folded  together  when  computers  are  used  only  to 
automate  practical  exercises.  See  following  text  for  further  explanation  how 
the  front  end  of  the  technology  lifecycle  is  shortened  in  practice. 

2  2 


Tvo  nev  costs  are  incurred  under  the  heading  of  operation  and 
maintenance:  computer  consumables  and  machine  maintenance.  These  tvo  items 
amount  to  a  little  less  than  $25  per  week  per  soldier,  vhich  is  a  little 
more  than  3X  of  the  weekly  cost  for  training  a  76C10  soldier.  This  might 
be  considered  a  small  but  significant  cost  increment  if  a  similar  hands-on 
system  were  implemented  in  the  basic  course.  Notice  that  costs  are 
computed  per  week  in  order  to  permit  cost  comparisons  between  courses  of 
different  length. 

Instructor  pay  and  allowances  require  a  bit  more  effort  to  evaluate. 

The  salient  question  concerns  the  ratio  of  instructors  to  students  -  does 

the  introduction  of  automated  hands-on  increase  or  decrease  instructor 
requirements?  Again  the  comparison  is  to  the  basic  76C10  course,  where 
between  65-83  instructors  teach  a  10  week  course  to  approximately  3000 
soldiers  each  year.  That  puts  about  600  trainees  in  classes  at  one  time, 
for  a  student/instructor  ratio  between  7-9.  In  TACCS/SAMS,  4  instructors 
teach  a  3-week  course  to  about  300  soldiers  each  year.  Thus,  about  18 
trainees  are  in  classes  at  one  time,  for  a  student/instructor  ratio  of  4.5. 
In  other  words,  automated  hands-on  appears  to  increase  instructor 
requirments.  The  additional  cost  per  student  per  week  amounts  to  about 
$50,  or  about  IX  of  the  weekly  training  cost  in  76C10. 

Comments  from  supply  department  staff  suggest  one  reason  why  greater 
classroom  supervision  may  be  needed.  For  hands-on  practice,  soldiers  are 
required  to  assemble  and  disassemble  workstations  as  they  would  do  in  the 
field.  Since  the  equipment  is  expensive  and  somewhat  fragile,  additional 
instructors  are  needed  to  reduce  breakage.  Although  this  added  instructor 
input  may  indeed  be  justified  in  order  to  effectively  train  automated  job 
tasks,  it  is  nonetheless  somewhat  ironic  that  computers,  with  their 
generally  acknowledged  instructional  potential,  should  increase  rather  than 
decrease  requirements  for  human  instructors.  It  should  also  be  noted  that 
the  extension  of  training  time  needed  to  teach  nev  automated  job  skills 
increases  training  costs  by  much  more  than  added  instructor  input.  Mainly 
because  of  this  extension  of  training  time,  this  first  introduction  of 
computers  into  the  76C10  classroom  substantially  increased  training  costs 
without  upgrading  existing  instructional  process  in  terms  of  more  efficient 
use  of  time. 


COST-EFFECTIVE  CLASSROOM  AUTOMATION 

Future  options  for  classroom  automation  and  ways  to  evaluate  their 
cost  effectiveness  can  now  be  considered  within  the  relevant  policy 
context.  The  latter  includes  not  only  current  automation  policy  but  also 
broader  fiscal  considerations  that  motivate  interest  in  cost  effectiveness 
in  the  first  place. 


7  3 


Incremental  Add-Ons  to  Hands-On  Software 


Current  policy  for  classroom  automation  at  the  QMS,  and  at  other  Army 
Schools,  makes  cost  effectiveness  analysis  unnecessary  for  basic 
investments  in  computer  equipment  and  appropriate  classroom  facilities. 
These  investments  will  be  made  and  used  to  automate  practical  exercises 
when  job  tasks  are  automated  in  the  field.  Vhile  this  policy  ignores  other 
instructional  uses  for  these  computers,  it  nevertheless  makes  additional 
uses  less  costly  since  at  least  the  compatible  hardvare  is  in  place 
already,  and  not  used  during  much  of  the  training  process.  Consequently, 
additional  instructional  uses  for  these  computers  need  incur  only 
incremental  software  costs. 

On  the  other  hand,  by  limiting  classroom  automation  to  hands-on 
simulation  of  job  tasks,  the  Quartermaster  School  and  other  Army  schools 
affirm  the  primary  instructional  value  of  lock-step  contact  between  an 
entire  class  and  the  platform  instructor.  In  the  same  way,  this  limited 
used  of  expensive  equipment  implies  doubt  about  more  advanced  or  extensive 
instructional  applications  of  computers,  or  at  least  an  unwillingness  to 
pay  the  associated  front  end  investment  costs. 

Taken  together,  the  existence  of  hands-on  workstations  and  the  implied 
skepticism  about  computer-based  instruction  suggest  that  classroom 
automation  beyond  hands-on  simulation  is  most  likely  to  prove  cost 
effective  in  small  increments.  These  might  be  called  "add-ons  to  hands-on 
training".  Because  they  would  capitalize  on  hardware  and  software 
investments  already  made,  and  on  the  growing  experience  of  using  computers 
as  adjuncts  to  existing  platform  instruction,  "add-ons"  would  cost  less 
than  "stand  alone"  options,  and  their  modest  instructional  objectives  could 
be  demonstrated  via  relatively  small  perturbations  in  existing 
instructional  process. 

Add-ons  to  hands-on  software  might  start  with  additional  practical 
exercises,  similar  to  those  already  programmed  by  subject  matter  experts 
for  hands-on  simulation,  but  some  exercises  could  be  easier  and  some  harder 
in  order  to  expand  the  amount  of  time  that  slower  and  faster  students  could 
learn  at  the  computer.  Add-ons  could  also  involve  more  elaborate 
programming  of  exercises,  increasing  their  instructional  value,  perhaps  via 
error  analysis,  more  timely  review  of  instructional  material,  or  more 
subtle  adaptation  of  problems  to  the  skill  level  of  the  trainee.  For  the 
soldiers  who  can  use  them,  such  changes  reduce  the  need  for  instructor 
supervision,  thus  giving  instructors  more  time  vith  those  who  need  human 
help  the  most. 

Needless  to  say,  these  add-ons  will  still  require  investment  dollars 
as  well  as  adjustments  in  current  classroom  procedures,  albeit  marginal 
amounts  of  both.  The  cost  effectiveness  methodology  proposed  above  is 
designed  to  identify  instructional  blocks  vhere  such  marginal  changes  may 
have  the  greatest  impact  on  training  time  and  thus  on  total  training  cost. 
However,  before  discussing  methodology  further,  attention  to  cost 
effectiveness  tradeoffs  may  be  motivated  by  relating  current  automation 
policy  to  the  recent  tightening  of  training  budgets. 
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Current  automation  policy  and  recent  classroom  experience  vlth 
computers,  as  far  as  they  go,  increase  training  cost.  Although  the 
preceding  cost  analysis  shoved  higher  cost  per  unit  of  training  time,  cost 
increases  occur  mainly  because  the  Course  has  been  (e.g.,  TACCS/SAMS)  and 
will  be  (e.g.,  ULC/ULLS)  extended  in  order  to  teach  new  automated  job 
tasks.  Training  improvements  that  have  or  will  result  from  automated 
hands-on  may  indeed  outweigh  their  costs,  but  in  an  era  of  tightening 
budgets,  any  added  cost  increases  fiscal  pressures  to  cut  costs  elsewhere, 
or  to  lower  training  standards.  In  fact,  in  FY88,  costs  for  the  76C10 
Course  are  being  cut  by  reducing.the  number  of  instructors  and  course 
development  staff  by  up  to  SOX. 


Such  cost  cutting  contrasts  with  the  strategy  proposed  at  the 
beginning  of  this  report  (See  Cost  Driver  and  Leverage,  above).  According 
to  that  cost  analysis,  the  most  powerful  cost  cutting  strategy  would 
shorten  the  length  of  training  because  Course  length  drives  over  95X  of 
training  costs.  In  fact,  cutting  back  inputs  to  technical  instruction  may 
increase  actual  (average)  training  time,  and  thus  defeat  the  larger  cost 
cutting  objective.  In  other  words,  cutting  back  on  technical  instruction 
may  run  the  leverage  argument  in  reverse.  Since  inputs  to  technical 
instruction  account  for  a  relatively  small  share  (14X)  of  total  training 
costs,  a  large  percentage  cut  here  accounts  for  a  much  smaller  percentage 
reduction  in  total  costs.  At  the  same  time,  as  long  as  training  standards 
are  maintained,  a  large  cutback  in  instructional  resources  may  slow  down 
the  training  process,  increasing  actual  training  time,  and  on  balance 
causing  an  increase  in  total  training  cost. 


Tailoring  Instruction  to  the  Subject  Hatter  and  to  the  Student 

An  alternative  approach  to  cost  reduction  would  target  hands-on 
add-ons  to  blocks  of  instruction  that  offer  the  most  attractive 
opportunities  to  shorten  training  time.  As  discussed  above,  these  blocks 
would  have  a  marginal  productivity  curves  with  right  tails  that  are 
relatively  long  and  low.  To  continue  the  comparison  between  the  two 
instruction  blocks  illustrated  in  Figure  5,  assume  that  the  right  tail  of 
marginal  productivity  in  block  {a]  runs  around  1-2X  of  the  class  during  the 
last  several  hours  of  instruction,  while  the  right  tail  in  block  [b]  runs 
around  8-10X.  Block  {a]  offers  the  more  attractive  opportunity  for 
hands-on  add-on  software  because  any  increase  in  the  rate  at  which 
soldiers  meet  the  criterion  will  permit  a  larger  reduction  in  training 
time,  while  holding  the  number  of  recycles  constant.  For  example,  if  the 
slowest  soldier  achieved  criterion  one  hour  earlier  in  both  blocks,  as  a 
result  of  greater  attention  from  the  instructor,  that  would  not  permit 
shortening  block  [bj  but  it  would  justify  chopping  one  hour  off  of  block 
[a]. 


10.  Personal  communication  with  Lev  Thayer,  Branch  Chief  in  charge  of  76C10 
training. 
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Furthermore,  this  targeting  of  hands-on  add-ons  enables  instructional 
inputs  to  exercise  maximum  economic  leverage  over  training  costs.  By 
reducing  actual  training  time  by  a  day,  for  all  classes  during  one  year, 
total  training  costs  can  be  reduced  by  about  $425,000  (see  calculation 
above).  If  this  were  accomplished  by  add-on  software,  and  if  the  latter 
had  a  useful  lifetime  of  3  years,  then  this  annual  savings  could  amortize 
over  a  million  dollar  investment.  Given  the  minimal  training  objective 

-  to  engage  the  bulk  of  students  in  activities  not  much  different  than 

vhat  they  vould  already  have  done  to  simulate  automated  job  tasks  -  such 

an  outlay  vould  seem  large.  But  even  if  a  million  dollars  were  invested, 
ltj  cost  saving  leverage  vould  also  be  great.  A  million  dollar  investment 
vould  increase  total  annual  training  costs  by  less  than  5X,  and  3-year 
total  training  costs  by  proportionally  less,  vhile  it  increases  annual 
inputs  to  direct  instruction  by  about  33*.  It  vould  seem  likely  that  such 
an  investment  in  instructional  softvare  could  account  for  more  than  a  2X 
reduction  in  current  training  time  (one  day  out  of  52  days). 

The  main  methodological  point  is  that  such  targeting  of  add-on 
technology,  to  instruction  blocks  vhere  it  can  have  the  greatest  impact  on 
training  time  and  cost,  provides  an  economic  rationale  for  tailoring 
instruction  and  instructional  technology  to  the  subject  matter  being 
trained.  Furthermore,  if  different  rates  of  learning  reflect  different 
instructional  needs,  then  this  approach  to  cost  effectiveness  also 
suggests  an  economic  rationale  for  tailoring  instruction  more  to  the  needs 
of  individual  students. 

There  is  a  recent  example  of  a  change  in  the  Course,  involving 
cooperative  learning,  that  indicates  a  villingness  by  Course  management 
and  staff  to  take  advantage  of  individual  differences  in  order  to  improve 
training  and  reduce  costs.  The  Training  Technology  Field  Activity  Program 
(TTFA)  at  the  Quartermaster  School  examined  the  effectiveness  of 
cooperative  learning  during  practical  exercises  in  the  76C10  Course. 
Cooperation  within  four-person  groups  reduces  instructor  burden  by 
enlisting  faster  or  more  motivated  students  to  ansver  questions  of  slower 
students  or  to  motivate  the  less  motivated.  As  discussed  in  Brooks  et  al. 
(1987),  recent  ARI  research  shows  that  cooperative  learning  in  the  76C10 
Course  may  reduce  the  number  of  recycles  vhile  having  no  measureable 
negative  effects  and  no  measureable  costs.  Of  course,  reduction  of 
recycles  actually  lovers  training  cost  by  decreasing  average  residence 
time  at  Fort  Lee.  Research  findings  vere  sufficient  that  Course 
management  recently  implemented  cooperative  learning  for  all  practical 
exercises  in  76C10. 

Another  reason  to  introduce  cooperative  learning  is  the  possible 
savings  in  equipment  permitted  vhen  several  soldiers  vould  use  the  same 
automated  vork  station  for  hands-on  training.  If  cooperative  learning 
vere  in  fact  used  for  automated  PEs,  then  the  resulting  more  efficient 
utilitization  of  computers  vould  permit  automation  of  instruction  beyond 
simulation  of  automated  job  tasks.  As  discussed  above,  the  QMS  school 
vill  soon  obtain  computers  sufficient  to  give  each  soldier  an  automated 
vorkstation  during  tvo  veeks  out  of  the  planned  12  veek  Course.  Although 
each  individual  doing  his  ovn  vork  is  definitely  the  Army's  preferred  mode 
for  hands-on  PE's  (i.e.  to  make  them  as  similar  to  job  experience  as 
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possible),  the  advantages  of  a  group  learning  approach,  as  cited  in  the 
above  research  vith  a  traditional  classroom,  are  potentially  greater  in  a 
computer-based  environment.  For  example,  if  it  could  be  shovn  that  a 
machine  could  be  used  just  as  veil  by  4  soldiers  at  one  time,  4  times  as 
many  classrooms  could  be  automated  (i.e.  12  instead  of  3),  vith  a 
resulting  substantial  increase  in  the  number  of  instruction  blocks  vhere 
computers  could  be  considered  as  tools  to  shorten  training  time. 


Implementation 

The  cost  effectiveness  methodology  presented  above  attempts  to  expand 
the  practical  domain  for  evaluation  of  computer  applications  in  Army 
classrooms.  Its  focus  on  training  as  a  production  process  highlights  the 
period  of  time,  tovard  the  end  of  an  instructional  block,  vhen 
productivity  is  relatively  lov  and  cost  per  soldier  relatively  high.  This 
is  also  the  period  of  time  vhen  current  policy  (for  classroom  automation) 
calls  for  the  application  of  computers  for  hands-on  simulation  of 
automated  job  tasks.  This  consistency  in  focus  permits  description  of 
future  automation  options  as  marginal  extensions  of  hands-on  technology. 

Current  automation  policy  lovers  costs  for  instructional  technology 
that  takes  advantage  of  hands-on  hardvare  and  softvare.  It  lovers  costs 
because  these  add-ons  to  hands-on  technology  need  incur  only  incremental 
costs  for  the  softvare  development.  The  same  add-on  technology  also 
entails  less  cost  for  experimental  research  to  test  its  effectiveness,  and 
less  disruption  of  normal  classroom  activities  to  conduct  necessary 
experiments,  because  the  nev  computer  applications  are  more  or  less 
marginal  extensions  of  vhat  is  already  being  done. 

The  cost  effectiveness  methodology  helps  to  plan  this  incremental 
development  process  by  targeting  instruction  blocks  and  hours  vithin  these 
blocks  that  offer  the  greatest  cost  savings  from  automation.  These 
savings  obtain  from  reduction  in  programmed  training  time  or  reduction  in 
recycles.  Both  changes  reduce  required  training  time,  although  the 
recycle  rate  and  the  related  dropout  rate  may  be  valued  as  instructional 
goals  quite  apart  from  cost.  Since  Course  length  drives  over  95X  of 
training  cost,  reduction  of  required  training  time  may  offset  costs 
incurred  to  expand  computer-based  instruction. 

This  potential  for  offsetting  automation  costs,  vith  the  time  and  cost 
savings  that  obtain  by  increasing  productivity,  recalls  the  analogy  to 
computer-based  equipment  simulators  (see  above).  The  latter  have 
demonstrated  cost  effectiveness  by  reducing  the  amount  of  time  that  must  be 
spent  using  more  expensive  actual  equipment.  Savings  resulting  from 
classroom  automation  involve  a  more  complex  series  of  adjustments  in 
training  process,  but  the  argument  is  made  that  the  current  relatively 
small  share  of  technical  training,  in  total  training  costs,  creates 
economic  leverage  for  nev  instructional  technology.  If  experimental 
research  shovs  that  nev  technology  can  reduce  training  time,  then  the 
methodology  can  target  investment  in  nev  technology  in  order  to  maximize 
savings  in  time  and  cost.  This  approach  to  cost  savings  should  be 
compared  to  present  attempts  to  meet  tightening  budget  constraints  by 
reducing  training  staff. 
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